Bee'dPGT/pro 13 jul 2004 
10/501428 

WO 03/061216 PCT/IB02/05501 

1 

SYSTEM FOR TRANSFERRING AND FILTERING VIDEO CONTENT DATA 



The invention relates to a method of transferring content data, in particular 
video, in which a sender provides the content data, the sender generates a transmission signal 
representing the content data, the transmission signal is transferred to a receiver via a 
transmission channel, a receiver receives the transmission signal and retrieves the content 
5 data, and the receiver renders the content data. 

The invention further relates to a device for sending content data, in particular 
video, which device comprises input means for providing the content data, and means for 
generating a transmission signal representing the content data. 

The invention further relates to a device for receiving content data, in 
10 particular video, which device comprises means for receiving a transmission signal, means 
for retrieving the content data from the transmission signal, and means for rendering the 
content data. 

The invention further relates to a signal and to a record carrier on which the 
signal is recorded. 

15 

Such methods, such devices, and such a signal are known from US 5,557,320. 
The document relates to a method of transferring content data in the form of a transmission 
signal from a sender to a receiver via a transmission channel in a sender-subscriber based, 

20 transmission traffic control system for video mail. The electronic data is transmitted 
primarily in the form of still or motion picture images, with or without audio track. A 
delivery network system provider offers a service in which the network has control over the 
actual time of video mail transmission. The path, bandwidth allocation, protocol format etc. 
can be set at the discretion of the network provider, and also compression/decompression or 

25 encryption processors can be provided. However for the user there is no flexibility for 
controlling the transmission of such content data. 
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Therefore it is an object of the invention to provide more flexible means for 
transferring content data. According to a first aspect of the invention this object is achieved 
with a method as in the opening paragraph, characterized in that the sender provides a filter 
for adapting the content data, the sender generates a filter signal representing the filter, the 
5 filter signal is transferred to a receiver via the transmission channel, the receiver receives the 
filter transmission signal and retrieves the filter, and the receiver processes the content data 
by the filter. 

According to a second aspect of the invention this object is achieved with a 
sending device as in the opening paragraph, characterized in that the device comprises filter 
10 input means for providing a filter for adapting the content data, and means for generating a 
filter signal representing the filter. 

According to a third aspect of the invention this object is achieved with a filter 
signal representing content data and a filter to be used in the first method, characterized in 
that the filter is for setting a remote processing unit for filtering the content data, in particular 
15 video. 

According to a fourth aspect of the invention this object is achieved with a 
rendering device as in the opening paragraph, characterized in that the device comprises 
means for receiving a filter signal representing a filter, and processing means for processing 
the content data as set by the filter. 

20 The invention is based on the following recognition. Devices, e.g. mobile 

phones, are becoming multimedia terminals which can send and receive video as well as 
audio and text messages. This invention addresses how somebody could edit and adapt a 
video clip they were sending to friends by attaching a pre-defined 'filter* which adds suitable 
effects to the clip (e.g. color adaptation, overlaying of icons, fade between en scenes, etc.). 

25 For example, if sending a video message back from a romantic holiday to your friends, you 
may like to edit the clip to give a more romantic atmosphere (music in the background, 'rose- 
tinted' filter, gentle fade in & out of video). However, if sending a message from an 
'adventure holiday*, you might like to adapt the clip with a lively filter (like an MTV music 
video). Instead of using the filter to edit the video clip before transmission, the filter is 

30 transmitted with the video, so allowing the receivers to view the clip with or without the 
filter, or use their own filters. 

According to a fifth aspect of the invention this object is achieved with a 
method of providing a filter to be used in the first method, characterized in that the filter is 
for setting a processing unit for filtering content data, in particular video, in which method a 
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number of filters is made available for selection, a client request is received indicative of a 
selection of a filter, a filter signal is generated representing the filter, and the filter signal is 
transferred to the client via a transmission channel. 

The effect is that a suitable filter can be selected and filters can be downloaded 
5 onto a sender or a receiver, without the filter itself being actually transferred between sender 
and receiver. 

Further preferred embodiments of the method, devices and signal according to 
the invention are given in the further claims. 

10 

These and other aspects of the invention will be apparent from and elucidated 
further with reference to the embodiments described by way of example in the following 
description and with reference to the accompanying drawings, in which 

Fig. 1 shows a sender and receiver, 
15 Fig. 2 shows a sender, receiver and filter provider, 

Fig. 3 shows video processing, 

Fig. 4 shows a record carrier, and 

Fig. 5 shows a filter. 

In the Figures, elements which correspond to elements already described have 
20 the same reference numerals. 



Fig. 1 shows a sender and receiver. The sender 1 1 has a content input unit 15 
for providing content data. The content data may for example be a video clip or a still image, 

25 retrieved from an external source or created within the sender by a camera module. The 
format of the input video can be any suitable, preferably standard, encoding (e.g. MPEG 
video). Further the content data may include text or audio, provided internally or from an 
external source under the control of the user at the sending side. The sender is provided with 
a filter input unit 16 for defining a filter to be used for adapting the content data to the liking 

30 of the user. The sender further comprises a content signal unit 17 for generating a 

transmission signal 13 representing the content data, and a filter signal unit 18 for generating 
a filter signal representing the filter as selected by the user for sending. In an embodiment the 
device has a filter creation unit for defining a specific filter by input provided by the user, 
whereby the user can define the parameters of the filter. The content data is encoded to a 
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transmission signal 13 by the content signal unit 17 and the filter data is encoded to a filter 
signal 14 by the filter signal unit 18. The signals are transferred to a receiver 12 via a 
transmission channel, e.g. a mobile phone network or internet, or a record carrier like a CD- 
R. The receiver 12 has a receiving signal unit 19 for receiving the transmission signal 13, and 
5 a filter input unit 20for receiving the filter signal 14. The receiver 12 further comprises a 
processing unit for performing the filter operation as defined by the filter signal as described 
below with reference to Figure 3. The content data is filtered and can be rendered, e.g. on a 
video display, or can be provided to an output. The filter as transferred by the filter signal 14 
can be provided with a range of effects, for example as follows. 
10 Colour filters: Alters the colour of each frame in a consistent manner, e.g.: 

Changes the whole clip to black and white 

Creates a graded rose-tinted filter (top of image very red, bottom of screen 
normal colour) 

Fades certain colours in and out in different frames. 
15 Image filters: Alters the structure of each frame, e.g.: 

Blurs edges. 

'Heat haze' effect, which wobbles parts of the image from side to side as the 
frames progress. 

'Hall of mirrors' effects — stretch some parts of image, shrink others. 
20 Overlay video/images. The operation is to draw icons on top of the video, or to 

have animated images moving across the screen, scrolling text at the bottom of the video, etc. 
Scene fades. Fade from one scene to another. 

The scene fade filter would require automatic scene change detection, and this 
implies that the filter must be of the 'non-streaming' type. 

25 For adapting audio or replacing an audio track the filter may contain audio 

filter operations, for example 'Add audio track' to add background music to the video clip. 
The format of the filter input file can be selected. An example format is given in the 
embodiment described with Figure 5 below. The filter operation is performed by a processing 
unit, which is set to the type of filtering as prescribed by the filter. The processing unit may 

30 take encoded video data as it is available from the content data input. In an embodiment the 
filter works on the decoded video, so it is not dependent on the video encoding format. In this 
way the filter processing can be used for different types of video data. There are many well- 
understood methods for filtering video to adapt the content (e.g. color/ contrast/ brightness as 
well as transitions between shots). 
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Fig. 2 shows a sender, receiver and filter provider. In addition to the system 
described with reference to Figure 1, the sender 1 1 now has a filter input unit for retrieving a 
filter from an external filter provider 22 via a network 21, e.g. the internet. Also the receiver 
has a filter input unit 25 for retrieving a filter from an external filter provider 22 via a 
5 network 21. The sender may select one of the candidate filters available at the provider and 
download the filter to his device. The filter may then be transferred to the receiver via the 
filter signal 24. Alternatively the sender my include just a reference to the filter in the filter 
signal 24, and the receiver then can retrieve the filter itself from the provider 22 or from a 
different external source. In an embodiment the receiver has a local storage for filter data for 

10 storing filters to be used more often. Now the filter indicated by reference may already be 
available in the receiver storage. Mobile phones currently allow some minor features to be 
downloaded over the network (often at a cost), e.g. new ringtones. Also downloading 
screensavers or applications (e.g. java applets) is well understood, e.g. for a PC or a Personal 
Digital Assistants PDA which can run a virtual machine. The downloading of a filter can be 

15 done using similar transmission techniques. 

Fig. 3 shows video processing. The video input is received as an encoded 
video signal in the video input unit 31 and transferred to a decoder 32. After decoding the 
decoded video is transferred to a processing unit 33. The processing unit is set to perform 
filter operations by a filter from a filter input unit 35, e.g. a filter input file received from an 

20 external source. The video after filtering is transferred to an output unit 34. In an embodiment 
the output unit includes a display for rendering the processed video. In a further embodiment 
the device has a selector 36 for selecting a video output signal after filtering or a video signal 
without filtering. A further selector may be included for using a user filter locally provided 
from a memory 37 in the device. It is noted that the video filter contains a general set of rules 

25 which can adapt any video clip with a suitable style. The filter is independent of any 

particular video clip. In particular the filter processing is applied to the decoded video, and is 
not dependent on the type of encoding. However, in a different embodiment the processing 
unit may be connected to the encoded video, and the filter may be applied to the encoded 
video. For the filter two main variants of processing can be used: streaming or non-streaming. 

30 In the streaming model, the filter engine adapts each video frame, and then outputs it before 
receiving the next frame. In other words, it does not require any information about future 
frames of video to perform the filtering process; this has the advantage that it adds minimal 
delay to the video display process. In the Non-Streaming model, the filtering process for a 
particular frame is dependent on future frames in the video stream. A simple example of this 
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case would be where the filter engine receives the whole decoded video clip, processes this 
(e.g. to find out where scene changes occur), and then outputs the filtered video clip. This 
allows more flexibility in the processing, but implies a delay in viewing the video. 

In an embodiment the video input unit 35 is able to provide a filter from an 
5 external source. The external source may be a record carrier to be read by a reading unit, e.g. 
a CD reader. Alternatively video filters could be downloaded by many different means to a 
device, e.g. a mobile handset. The invention assumes that the mobile device has the ability to 
connect to a network using standard (internet-based) technology. If this is the case, each 
video filter could be stored on the network with its own URL (Universal Resource Locator) 

1 0 the handset would then request this URL for download. The handset would be able to identify 
the downloaded file as a 'video filter file' as these files would be assigned their own unique 
data type identifying them as video filters (e.g. if the download is via HTTP, the MIME-type 
is transmitted automatically with the data). It is noted that the video input unit 3 1 and the 
filter input unit 35 may be configured to receive a combined video & filter message from a 

15 sender. Transmission of data messages between two mobile devices is well understood (e.g. 
text messages like SMS). Further the email transmission between a sender and a receiver is 
well known, and could also be arranged between 2 internet-enabled mobile phones. For a 
combined video & filter message to be transmitted, it would be necessary to transmit the two 
components of the message together, along with their respective data types. For example, if 

20 using email as the underlying transport protocol, the message could be transmitted as a 

multipart MIME message; the receiving device would then receive the video and the filter at 
the same time, and display the filtered video accordingly. In an embodiment the sender and 
receiver can access data via URLs on the network by the well known HTTP protocol, e.g. 
using a browser, and receive & transmit email messages, e.g. via POP3/SMTP, and decode 

25 and display MPEG4 video. For example 2 WAP-enabled mobile phones (Wireless Access 
Protocol) can access data using the WTP protocol. Further the sender may include a video 
content generator, e.g. a camera for capture and an encoder for encoding video as MPEG4. 
Assuming this, the invention could be implemented using the following steps. Filters are 
downloaded via a browser over HTTP. They have a well-defined MIME-type (e.g. 'text- 

30 video-filter*). The filter file format is XML-based; an example filter is shown below. On 

download, the filter is stored in the mobile phone memory. When transmitting the combined 
video & filter, the message is sent as a multipart-MIME email message (containing MPEG4 
clip & Hext-video-filter' filter). Using these concepts, the development of a filter would then 
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only be dependent on how the software for creating and displaying the video message is 
defined, and how the 'filter engine' is defined. 

Fig. 4 shows a record carrier 41 provided with a track 42 intended for 
recording and a central hole 40. The track 42 is arranged in accordance with a spiral pattern 
5 of windings. A first part of the track in the middle area of the record carrier constitutes a 
Lead-In 43 for accommodating control data. The record carrier comprises a transparent 
substrate with a recording layer and a protective layer. The recording layer may be optically 
writable, for example via phase change, or magneto-optically writable by a device for writing 
information such as the known CD-Rewritable or CD-Recordable. The recording layer may 

10 also be provided with information via a production process, in which first a master disc is 
made which is subsequently multiplied through pressing. The information is organized in 
information blocks and is represented by optically readable marks in the form of a succession 
of areas reflecting much radiation and little radiation such as, for example a succession of pits 
of different lengths in a CD. The information blocks comprise a filter signal representing the 

15 filter data structure and parameters as described above. An embodiment of the record carrier 
comprises a large collection of filters, so that the user may select a suitable filter. In an 
embodiment the record carrier comprises video processing software for programming a 
standard processing unit, e.g. a PC, for performing the video encoding or decoding and filter 
processing. 

20 Fig. 5 shows a filter. The filter 5 1 is defined in the XML type language 

(extended Markup Language). The first instruction indicates a colour filter, which is to be 
retrieved from an external source; in this case an internet URL is indicated. This part of the 
filter will change the colour to black and white. The next section may contain image 
operations, but non are included in this example. The last section contains an overlay 

25 operation, in this case an icon named smiley is to be overlaid by the processing unit. 

Although the invention has been explained by embodiments using mobile 
handsets, any rendering device or other suitable device may be used. Also for the record 
carrier an optical disc has been described, but other media, such as a magnetic disc or tape, 
may be used. It is noted, that in this document the word 'comprising* does not exclude the 

30 presence of other elements or steps than those listed and the word 'a* or 'an* preceding an 
element does not exclude the presence of a plurality of such elements, that any reference 
signs do not limit the scope of the claims, that the invention maybe implemented by means 
of both hardware and software, and that several 'means' may be represented by the same item 



WO 03/061216 PCT/IB02/05501 

8 

of hardware. Further, the scope of the invention is not limited to the embodiments, and the 
invention lies in each and every novel feature or combination of features described above. 



